Functional diversity of dopamine axons in prefrontal cortex during classical conditioning.

Midbrain dopamine neurons impact neural processing in the prefrontal cortex (PFC) through mesocortical projections. However, the signals conveyed by dopamine projections to the PFC remain unclear, particularly at the single-axon level. Here, we investigated dopaminergic axonal activity in the medial PFC (mPFC) during reward and aversive processing. By optimizing microprism-mediated two-photon calcium imaging of dopamine axon terminals, we found diverse activity in dopamine axons responsive to both reward and aversive stimuli. Some axons exhibited a preference for reward, while others favored aversive stimuli, and there was a strong bias for the latter at the population level. Long-term longitudinal imaging revealed that the preference was maintained in reward- and aversive-preferring axons throughout classical conditioning in which rewarding and aversive stimuli were paired with preceding auditory cues. However, as mice learned to discriminate reward or aversive cues, a cue activity preference gradually developed only in aversive-preferring axons. We inferred the trial-by-trial cue discrimination based on machine learning using anticipatory licking or facial expressions, and found that successful discrimination was accompanied by sharper selectivity for the aversive cue in aversive-preferring axons. Our findings indicate that a group of mesocortical dopamine axons encodes aversive-related signals, which are modulated by both classical conditioning across days and trial-by-trial discrimination within a day.

The basal forebrain to lateral habenula circuitry mediates social behavioral maladaptation.

Elucidating the neural basis of fear allows for more effective treatments for maladaptive fear often observed in psychiatric disorders. Although the basal forebrain (BF) has an essential role in fear learning, its function in fear expression and the underlying neuronal and circuit substrates are much less understood. Here we report that BF glutamatergic neurons are robustly activated by social stimulus following social fear conditioning in male mice. And cell-type-specific inhibition of those excitatory neurons largely reduces social fear expression. At the circuit level, BF glutamatergic neurons make functional contacts with the lateral habenula (LHb) neurons and these connections are potentiated in conditioned mice. Moreover, optogenetic inhibition of BF-LHb glutamatergic pathway significantly reduces social fear responses. These data unravel an important function of the BF in fear expression via its glutamatergic projection onto the LHb, and suggest that selective targeting BF-LHb excitatory circuitry could alleviate maladaptive fear in relevant disorders.

Central amygdala contributes to stimulus facilitation and pre-stimulus vigilance during cerebellar learning.

Our previous studies found that the central amygdala (CeA) modulates cerebellum-dependent eyeblink conditioning (EBC) using muscimol inactivation. We also found that CeA inactivation decreases cerebellar neuronal activity during the conditional stimulus (CS) from the start of training. Based on these findings, we hypothesized that the CeA facilitates CS input to the cerebellum. The current study tested the CS facilitation hypothesis using optogenetic inhibition with archaerhodopsin (Arch) and excitation with channelrhodopsin (ChR2) of the CeA during EBC in male rats. Optogenetic manipulations were administered during the 400 ms tone CS or during a 400 ms pre-CS period. As predicted by the CS facilitation hypothesis CeA inhibition during the CS impaired EBC and CeA excitation during the CS facilitated EBC. Unexpectedly, CeA inhibition just prior to the CS also impaired EBC, while CeA excitation during the pre-CS pathway did not facilitate EBC. The results suggest that the CeA contributes to CS facilitation and vigilance during the pre-CS period. These putative functions of the CeA may be mediated through separate output pathways from the CeA to the cerebellum.

Changes in responses of the amygdala and hippocampus during fear conditioning are associated with persecutory beliefs.

The persecutory delusion is the most common symptom of psychosis, yet its underlying neurobiological mechanisms are poorly understood. Prior studies have suggested that abnormalities in medial temporal lobe-dependent associative learning may contribute to this symptom. In the current study, this hypothesis was tested in a non-clinical sample of young adults without histories of psychiatric treatment (n = 64), who underwent classical Pavlovian fear conditioning while fMRI data were collected. During the fear conditioning procedure, participants viewed images of faces which were paired (the CS+) or not paired (the CS-) with an aversive stimulus (a mild electrical shock). Fear conditioning-related neural responses were measured in two medial temporal lobe regions, the amygdala and hippocampus, and in other closely connected brain regions of the salience and default networks. The participants without persecutory beliefs (n = 43) showed greater responses to the CS- compared to the CS+ in the right amygdala and hippocampus, while the participants with persecutory beliefs (n = 21) failed to exhibit this response. These between-group differences were not accounted for by symptoms of depression, anxiety or a psychosis risk syndrome. However, the severity of subclinical psychotic symptoms overall was correlated with the level of this aberrant response in the amygdala (p = .013) and hippocampus (p = .033). Thus, these findings provide evidence for a disruption of medial temporal lobe-dependent associative learning in young people with subclinical psychotic symptoms, specifically persecutory thinking.

Distinct Hippocampal Oscillation Dynamics in Trace Eyeblink Conditioning Task for Retrieval and Consolidation of Associations.

Trace eyeblink conditioning (TEBC) has been widely used to study associative learning in both animals and humans. In this paradigm, conditioned responses (CRs) to conditioned stimuli (CS) serve as a measure for retrieving learned associations between the CS and the unconditioned stimuli (US) within a trial. Memory consolidation, that is, learning over time, can be quantified as an increase in the proportion of CRs across training sessions. However, how hippocampal oscillations differentiate between successful memory retrieval within a session and consolidation across TEBC training sessions remains unknown. To address this question, we recorded local field potentials (LFPs) from the rat dorsal hippocampus during TEBC and investigated hippocampal oscillation dynamics associated with these two functions. We show that transient broadband responses to the CS were correlated with memory consolidation, as indexed by an increase in CRs across TEBC sessions. In contrast, induced alpha (8-10 Hz) and beta (16-20 Hz) band responses were correlated with the successful retrieval of the CS-US association within a session, as indexed by the difference in trials with and without CR.

Conditioning and pseudoconditioning differently change intrinsic excitability of inhibitory interneurons in the neocortex.

Many studies indicate a broad role of various classes of GABAergic interneurons in the processes related to learning. However, little is known about how the learning process affects intrinsic excitability of specific classes of interneurons in the neocortex. To determine this, we employed a simple model of conditional learning in mice where vibrissae stimulation was used as a conditioned stimulus and a tail shock as an unconditioned one. In vitro whole-cell patch-clamp recordings showed an increase in intrinsic excitability of low-threshold spiking somatostatin-expressing interneurons (SST-INs) in layer 4 (L4) of the somatosensory (barrel) cortex after the conditioning paradigm. In contrast, pseudoconditioning reduced intrinsic excitability of SST-LTS, parvalbumin-expressing interneurons (PV-INs), and vasoactive intestinal polypeptide-expressing interneurons (VIP-INs) with accommodating pattern in L4 of the barrel cortex. In general, increased intrinsic excitability was accompanied by narrowing of action potentials (APs), whereas decreased intrinsic excitability coincided with AP broadening. Altogether, these results show that both conditioning and pseudoconditioning lead to plastic changes in intrinsic excitability of GABAergic interneurons in a cell-specific manner. In this way, changes in intrinsic excitability can be perceived as a common mechanism of learning-induced plasticity in the GABAergic system.

An analysis of reinstatement after extinction of a conditioned taste aversion.

Taste aversion learning has sometimes been considered a specialized form of learning. In several other conditioning preparations, after a conditioned stimulus (CS) is conditioned and extinguished, reexposure to the unconditioned stimulus (US) by itself can reinstate the extinguished conditioned response. Reinstatement has been widely studied in fear and appetitive Pavlovian conditioning, as well as operant conditioning, but its status in taste aversion learning is more controversial. Six taste-aversion experiments with rats therefore sought to discover conditions that might encourage it there. The results often yielded little to no evidence of reinstatement, and we also found no evidence of concurrent recovery, a related phenomenon in which responding to a CS that has been conditioned and extinguished is restored if a second CS is separately conditioned. However, a key result was that reinstatement occurred when the conditioning procedure involved multiple closely spaced conditioning trials that could have allowed the animal to learn that a US presentation signaled or set the occasion for another trial with a US. Such a mechanism is precluded in many taste aversion experiments because they often use very few conditioning trials. Overall, the results suggest that taste aversion learning is experimentally unique, though not necessarily biologically or evolutionarily unique. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Bayesian reinforcement learning: A basic overview.

We and other animals learn because there is some aspect of the world about which we are uncertain. This uncertainty arises from initial ignorance, and from changes in the world that we do not perfectly know; the uncertainty often becomes evident when our predictions about the world are found to be erroneous. The Rescorla-Wagner learning rule, which specifies one way that prediction errors can occasion learning, has been hugely influential as a characterization of Pavlovian conditioning and, through its equivalence to the delta rule in engineering, in a much wider class of learning problems. Here, we review the embedding of the Rescorla-Wagner rule in a Bayesian context that is precise about the link between uncertainty and learning, and thereby discuss extensions to such suggestions as the Kalman filter, structure learning, and beyond, that collectively encompass a wider range of uncertainties and accommodate a wider assortment of phenomena in conditioning.

Mesoscale simulations predict the role of synergistic cerebellar plasticity during classical eyeblink conditioning.

According to the motor learning theory by Albus and Ito, synaptic depression at the parallel fibre to Purkinje cells synapse (pf-PC) is the main substrate responsible for learning sensorimotor contingencies under climbing fibre control. However, recent experimental evidence challenges this relatively monopolistic view of cerebellar learning. Bidirectional plasticity appears crucial for learning, in which different microzones can undergo opposite changes of synaptic strength (e.g. downbound microzones-more likely depression, upbound microzones-more likely potentiation), and multiple forms of plasticity have been identified, distributed over different cerebellar circuit synapses. Here, we have simulated classical eyeblink conditioning (CEBC) using an advanced spiking cerebellar model embedding downbound and upbound modules that are subject to multiple plasticity rules. Simulations indicate that synaptic plasticity regulates the cascade of precise spiking patterns spreading throughout the cerebellar cortex and cerebellar nuclei. CEBC was supported by plasticity at the pf-PC synapses as well as at the synapses of the molecular layer interneurons (MLIs), but only the combined switch-off of both sites of plasticity compromised learning significantly. By differentially engaging climbing fibre information and related forms of synaptic plasticity, both microzones contributed to generate a well-timed conditioned response, but it was the downbound module that played the major role in this process. The outcomes of our simulations closely align with the behavioural and electrophysiological phenotypes of mutant mice suffering from cell-specific mutations that affect processing of their PC and/or MLI synapses. Our data highlight that a synergy of bidirectional plasticity rules distributed across the cerebellum can facilitate finetuning of adaptive associative behaviours at a high spatiotemporal resolution.

Reduced reactivity to fear conditioning and pain tests in persons involved in violent video gaming is influenced by adverse childhood experiences.

Video gaming, including violent video gaming, has become very common and lockdown measures of the COVID-19 pandemic even increased the prevalence rates. In this study, we examined if violent video gaming is associated with more adverse childhood experiences (ACE) and if it impairs pain processing and fear conditioning. We tested three groups of participants (violent video gamers, nonviolent video gamers, and non-gamers) and examined fear conditioning as well as pain perception during functional magnetic resonance imaging (fMRI). Violent video gamers displayed significantly higher pain thresholds as well as pain tolerance for electric stimulation, pressure pain stimulation, and cold pressor pain measurements than nonviolent video gamers and non-gamers. This relationship was moderated by adverse childhood experiences, especially physical neglect. Brain images acquired during the fear conditioning fMRI task showed that violent video gamers display significantly less differential brain activation to stimuli signaling pain versus no pain in the anterior cingulate cortex, the juxtapositional lobule cortex, and the paracingulate gyrus compared to non-gamers. There was also a significant negative correlation between adverse childhood experiences and activation in the precuneus and the intracalcarine cortex for signals of pain versus safety. The results of this study imply that violent video gaming is related to reduced processing of pain and signals of pain in a fear learning task, dependent of adverse childhood experiences. These mechanisms need to be examined in more detail and these data could be helpful in preventing the onset and adverse consequences of violent video gaming.

[Learning and Neural Activity: Beyond Localization].

Learning is classified into two types: "classical conditioning," which modifies simple reflexes, and "operant conditioning," which modifies complex voluntary behaviors. The neural circuits underlying these two types differ significantly. During the learning process of operant conditioning tasks, various changes in firing rate and firing synchrony of neurons can be observed across multiple brain regions. Additionally, neuronal firing rate and synchrony in several brain regions can be voluntarily controlled through operant conditioning. Consequently, it is evident that neurons in widespread brain regions have the potential for plastic changes to facilitate learning. It may be suggested that the learning of complex voluntary behaviors is underlined by widespread dynamic changes in neural activity and is not restricted to only a few brain regions.

Auditory aversive generalization learning prompts threat-specific changes in alpha-band activity.

Pairing a neutral stimulus with aversive outcomes prompts neurophysiological and autonomic changes in response to the conditioned stimulus (CS+), compared to cues that signal safety (CS-). One of these changes-selective amplitude reduction of parietal alpha-band oscillations-has been reliably linked to processing of visual CS+. It is, however, unclear to what extent auditory conditioned cues prompt similar changes, how these changes evolve as learning progresses, and how alpha reduction in the auditory domain generalizes to similar stimuli. To address these questions, 55 participants listened to three sine wave tones, with either the highest or lowest pitch (CS+) being associated with a noxious white noise burst. A threat-specific (CS+) reduction in occipital-parietal alpha-band power was observed similar to changes expected for visual stimuli. No evidence for aversive generalization to the tone most similar to the CS+ was observed in terms of alpha-band power changes, aversiveness ratings, or pupil dilation. By-trial analyses found that selective alpha-band changes continued to increase as aversive conditioning continued, beyond when participants reported awareness of the contingencies. The results support a theoretical model in which selective alpha power represents a cross-modal index of continuous aversive learning, accompanied by sustained sensory discrimination of conditioned threat from safety cues.

Cardiorespiratory rhythm-contingent trace eyeblink conditioning in elderly adults.

Learning outcome is modified by the degree to which the subject responds and pays attention to specific stimuli. Our recent research suggests that presenting stimuli in contingency with a specific phase of the cardiorespiratory rhythm might expedite learning. Specifically, expiration-diastole (EXP-DIA) is beneficial for learning trace eyeblink conditioning (TEBC) compared with inspiration-systole (INS-SYS) in healthy young adults. The aim of this study was to investigate whether the same holds true in healthy elderly adults (n = 50, aged >70 yr). Participants were instructed to watch a silent nature film while TEBC trials were presented at either INS-SYS or EXP-DIA (separate groups). Learned responses were determined as eyeblinks occurring after the tone conditioned stimulus (CS), immediately preceding the air puff unconditioned stimulus (US). Participants were classified as learners if they made at least five conditioned responses (CRs). Brain responses to the stimuli were measured by electroencephalogram (EEG). Memory for the film and awareness of the CS-US contingency were evaluated with a questionnaire. As a result, participants showed robust brain responses to the CS, acquired CRs, and reported awareness of the CS-US relationship to a variable degree. There was no difference between the INS-SYS and EXP-DIA groups in any of the above. However, when only participants who learned were considered, those trained at EXP-DIA (n = 11) made more CRs than those trained at INS-SYS (n = 13). Thus, learned performance could be facilitated in those elderly who learned. However, training at a specific phase of cardiorespiratory rhythm did not increase the proportion of participants who learned.NEW & NOTEWORTHY We trained healthy elderly individuals in trace eyeblink conditioning, either at inspiration-systole or at expiration-diastole. Those who learned exhibited more conditioned responses when trained at expiration-diastole rather than inspiration-systole. However, there was no difference between the experimental groups in the proportion of individuals who learned or did not learn.

CK2 negatively regulates the extinction of remote fear memory.

Cognitive behavioral therapy, rooted in exposure therapy, is currently the primary approach employed in the treatment of anxiety-related conditions, including post-traumatic stress disorder (PTSD). In laboratory settings, fear extinction in animals is a commonly employed technique to investigate exposure therapy; however, the precise mechanisms underlying fear extinction remain elusive. Casein kinase 2 (CK2), which regulates neuroplasticity via phosphorylation of its substrates, has a significant influence in various neurological disorders, such as Alzheimer's disease and Parkinson's disease, as well as in the process of learning and memory. In this study, we adopted a classical Pavlovian fear conditioning model to investigate the involvement of CK2 in remote fear memory extinction and its underlying mechanisms. The results indicated that the activity of CK2 in the medial prefrontal cortex (mPFC) of mice was significantly upregulated after extinction training of remote cued fear memory. Notably, administration of the CK2 inhibitor CX-4945 prior to extinction training facilitated the extinction of remote fear memory. In addition, CX-4945 significantly upregulated the expression of p-ERK1/2 and p-CREB in the mPFC. Our results suggest that CK2 negatively regulates remote fear memory extinction, at least in part, by inhibiting the ERK-CREB pathway. These findings contribute to our understanding of the underlying mechanisms of remote cued fear extinction, thereby offering a theoretical foundation and identifying potential targets for the intervention and treatment of PTSD.

The ventromedial prefrontal cortex in response to threat omission is associated with subsequent explicit safety memory.

In order to memorize and discriminate threatening and safe stimuli, the processing of the actual absence of threat seems crucial. Here, we measured brain activity with fMRI in response to both threat conditioned stimuli and their outcomes by combining threat learning with a subsequent memory paradigm. Participants (N = 38) repeatedly saw a variety of faces, half of which (CS+) were associated with an aversive unconditioned stimulus (US) and half of which were not (CS-). When an association was later remembered, the hippocampus had been more active (than when forgotten). However, the ventromedial prefrontal cortex predicted subsequent memory specifically during safe associations (CS- and US omission responses) and the left dorsolateral prefrontal cortex during outcomes in general (US and US omissions). In exploratory analyses of the theoretically important US omission, we found extended involvement of the medial prefrontal cortex and an enhanced functional connectivity to visual and somatosensory cortices, suggesting a possible function in sustaining sensory information for an integration with semantic memory. Activity in visual and somatosensory cortices together with the inferior frontal gyrus also predicted memory performance one week after learning. The findings imply the importance of a close interplay between prefrontal and sensory areas during the processing of safe outcomes-or 'nothing'-to establish declarative safety memory.

The effects of a retrieval cue on renewal of conditioned responses in human appetitive conditioning.

Contextual renewal of reward anticipation may be one potential mechanism underlying relapse in eating and substance use disorders. We therefore tested retrieval cues, a method derived from an inhibitory retrieval-based model of extinction learning to attenuate contextual renewal using an appetitive conditioning paradigm. A pilot study was carried out in Experiment 1 to validate a differential chocolate conditioning paradigm, in which a specific tray was set up as a conditioned stimulus (CS) for eating chocolate (unconditioned stimulus, US). Using an ABA renewal design in Experiment 2, half of the participants were presented with a retrieval cue in the acquisition phase (group AC) and the other half in the extinction phase (group EC). Presentation of the retrieval cue in the EC was associated with reduced renewal of US-expectancy, while there was a clear renewal effect for US-expectancy in the AC. One limitation was the difference in cue presentations between both groups due to the number of trials in acquisition and extinction. Experiment 3 therefore aimed at replicating the results of Experiment 2, but with fewer cue presentations for the EC to match the AC. No significant group differences were observed indicating no effect of the retrieval cue. Theoretical and clinical implications in light of the differing results are discussed.

Adolescents flexibly adapt action selection based on controllability inferences.

From early in life, we encounter both controllable environments, in which our actions can causally influence the reward outcomes we experience, and uncontrollable environments, in which they cannot. Environmental controllability is theoretically proposed to organize our behavior. In controllable contexts, we can learn to proactively select instrumental actions that bring about desired outcomes. In uncontrollable environments, Pavlovian learning enables hard-wired, reflexive reactions to anticipated, motivationally salient events, providing "default" behavioral responses. Previous studies characterizing the balance between Pavlovian and instrumental learning systems across development have yielded divergent findings, with some studies observing heightened expression of Pavlovian learning during adolescence and others observing a reduced influence of Pavlovian learning during this developmental stage. In this study, we aimed to investigate whether a theoretical model of controllability-dependent arbitration between learning systems might explain these seemingly divergent findings in the developmental literature, with the specific hypothesis that adolescents' action selection might be particularly sensitive to environmental controllability. To test this hypothesis, 90 participants, aged 8-27, performed a probabilistic-learning task that enables estimation of Pavlovian influence on instrumental learning, across both controllable and uncontrollable conditions. We fit participants' data with a reinforcement-learning model in which controllability inferences adaptively modulate the dominance of Pavlovian versus instrumental control. Relative to children and adults, adolescents exhibited greater flexibility in calibrating the expression of Pavlovian bias to the degree of environmental controllability. These findings suggest that sensitivity to environmental reward statistics that organize motivated behavior may be heightened during adolescence.

Pupillary Responses Reflect Dynamic Changes in Multiple Cognitive Factors During Associative Learning in Primates.

Associative learning involves complex interactions of multiple cognitive factors. While adult subjects can articulate these factors verbally, for model animals such as macaques, we rely on behavioral outputs. In our study, we used pupillary responses as an alternative measure to capture these underlying cognitive changes. We recorded the dynamic changes in the pupils of three male macaques when they learned the associations between visual stimuli and reward sizes under the classical Pavlovian experimental paradigm. We found that during the long-term learning process, the gradual changes in the pupillary response reflect the changes in the cognitive state of the animals. The pupillary response can be explained by a linear combination of components corresponding to multiple cognitive factors. These components reflect the impact of visual stimuli on the pupils, the prediction of reward values associated with the visual stimuli, and the macaques' understanding of the current experimental reward rules. The changing patterns of these factors during interday and intraday learning clearly demonstrate the enhancement of current reward-stimulus association and the weakening of previous reward-stimulus association. Our study shows that the dynamic response of pupils can serve as an objective indicator to characterize the psychological changes of animals, understand their learning process, and provide important tools for exploring animal behavior during the learning process.

Variation in the effectiveness of reinforcement and nonreinforcement in generating different conditioned behaviors.

Rat autoshaping procedures generate two readily measurable conditioned responses: During lever presentations that have previously signaled food, rats approach the food well (called goal-tracking) and interact with the lever itself (called sign-tracking). We investigated how reinforced and nonreinforced trials affect the overall and temporal distributions of these two responses across 10-second lever presentations. In two experiments, reinforced trials generated more goal-tracking than sign-tracking, and nonreinforced trials resulted in a larger reduction in goal-tracking than sign-tracking. The effect of reinforced trials was evident as an increase in goal-tracking and reduction in sign-tracking across the duration of the lever presentations, and nonreinforced trials resulted in this pattern transiently reversing and then becoming less evident with further training. These dissociations are consistent with a recent elaboration of the Rescorla-Wagner model, HeiDI (Honey, R.C., Dwyer, D.M., & Iliescu, A.F. (2020a). HeiDI: A model for Pavlovian learning and performance with reciprocal associations. Psychological Review, 127, 829-852.), a model in which responses related to the nature of the unconditioned stimulus (e.g., goal-tracking) have a different origin than those related to the nature of the conditioned stimulus (e.g., sign-tracking).

Excitability mediates allocation of pre-configured ensembles to a hippocampal engram supporting contextual conditioned threat in mice.

Little is understood about how engrams, sparse groups of neurons that store memories, are formed endogenously. Here, we combined calcium imaging, activity tagging, and optogenetics to examine the role of neuronal excitability and pre-existing functional connectivity on the allocation of mouse cornu ammonis area 1 (CA1) hippocampal neurons to an engram ensemble supporting a contextual threat memory. Engram neurons (high activity during recall or TRAP2-tagged during training) were more active than non-engram neurons 3 h (but not 24 h to 5 days) before training. Consistent with this, optogenetically inhibiting scFLARE2-tagged neurons active in homecage 3 h, but not 24 h, before conditioning disrupted memory retrieval, indicating that neurons with higher pre-training excitability were allocated to the engram. We also observed stable pre-configured functionally connected sub-ensembles of neurons whose activity cycled over days. Sub-ensembles that were more active before training were allocated to the engram, and their functional connectivity increased at training. Therefore, both neuronal excitability and pre-configured functional connectivity mediate allocation to an engram ensemble.